1 00:00:00,000 --> 00:00:14,080 This is Hacker Public Radio episode 4,053 for Wednesday, the 14th of February 2024. 2 00:00:14,080 --> 00:00:19,000 Today's show is entitled, third party integrations. 3 00:00:19,000 --> 00:00:22,960 It is hosted by Lee and is about 12 minutes long. 4 00:00:22,960 --> 00:00:25,560 It carries a clean flag. 5 00:00:25,560 --> 00:00:40,560 The summary is, connecting your application with others. 6 00:00:40,560 --> 00:00:44,560 Hi, I'm Lee and today I'm going to talk about connecting things. 7 00:00:44,560 --> 00:00:50,560 Specifically connecting an application we've developed with some other company's application. 8 00:00:50,560 --> 00:00:53,560 Although this might be an application made for our own use, 9 00:00:53,560 --> 00:00:57,560 there's a few bits one that's been provided for other people to use. 10 00:00:57,560 --> 00:01:03,560 Suppose we've created an application that has a feature that produces invoices. 11 00:01:03,560 --> 00:01:09,560 But the end users do their company counts in their favourite accounts package, such as Sage, QuickBooks or Zero. 12 00:01:09,560 --> 00:01:15,560 These users will then have to duplicate the effort of producing an invoice in our application, 13 00:01:15,560 --> 00:01:22,560 because they then also have to create the same invoice in their own accounts package, just to keep their account and happy. 14 00:01:22,560 --> 00:01:25,560 When it be nice if the invoices they've produced in our application, 15 00:01:25,560 --> 00:01:31,560 look at the same time magically appear in the bookkeeping system used by their accountants. 16 00:01:31,560 --> 00:01:34,560 Well that's what I'm going to talk about. 17 00:01:34,560 --> 00:01:42,560 So, how could we go about solving the problem of transferring information from one application to another? 18 00:01:42,560 --> 00:01:48,560 A simple approach would be to export some kind of data from the source application into a file, 19 00:01:48,560 --> 00:01:52,560 then import this same file into the destination application. 20 00:01:52,560 --> 00:01:57,560 This would be a good approach, but it has some drawbacks. 21 00:01:57,560 --> 00:02:00,560 Firstly, it's a manual process. 22 00:02:00,560 --> 00:02:07,560 Every time the user wants to transfer some information, they have to go through this process of export and imports. 23 00:02:07,560 --> 00:02:11,560 Secondly, it only works in one direction at a time. 24 00:02:11,560 --> 00:02:15,560 An invoice generally contains the contact details of the customer, 25 00:02:15,560 --> 00:02:19,560 but so we don't confuse two people who are both called James Smith. 26 00:02:19,560 --> 00:02:23,560 The accounts package might give each customer reference number. 27 00:02:23,560 --> 00:02:29,560 It'd be useful to be able to find out the customer's reference number from the accounts package, 28 00:02:29,560 --> 00:02:33,560 but this could be filled in when exporting the invoice from that application. 29 00:02:33,560 --> 00:02:39,560 But to do that, we'd first have to export the customers from the accounts package into our application 30 00:02:39,560 --> 00:02:43,560 before exporting the invoices from our program back into the accounts package. 31 00:02:44,560 --> 00:02:46,560 And that might not be the end of it. 32 00:02:46,560 --> 00:02:51,560 What if we also wanted to cross reference part number stored in the accounts package? 33 00:02:51,560 --> 00:02:56,560 This simple process has now become a nightmare of back and forth exports. 34 00:02:56,560 --> 00:03:04,560 It really would be helpful if the transfer of information was a conversation rather than the one-way monologue. 35 00:03:04,560 --> 00:03:07,560 So, instead of doing a simple one-way export, 36 00:03:07,560 --> 00:03:12,560 there are cases where it would be better to let the two applications talk to each other directly, 37 00:03:12,560 --> 00:03:15,560 using a standard protocol of some kind. 38 00:03:15,560 --> 00:03:20,560 So, let's tell our users to give us the password of their accounts package, 39 00:03:20,560 --> 00:03:25,560 so our application can log into it on their behalf in order to do the transfer. 40 00:03:25,560 --> 00:03:27,560 How does that sound? 41 00:03:27,560 --> 00:03:30,560 Actually, this is horrible for security. 42 00:03:30,560 --> 00:03:36,560 Even if our users trust us with unrestricted access to every aspect of their finances 43 00:03:36,560 --> 00:03:41,560 from the payroll details of their employees to whether they've paid their taxes, 44 00:03:41,560 --> 00:03:45,560 this still means that their password is no longer under their control. 45 00:03:45,560 --> 00:03:49,560 If we suffer a later breach, so do all our users. 46 00:03:49,560 --> 00:03:56,560 So, how can the user let our application have access to a limited subset of their data in a third-party application 47 00:03:56,560 --> 00:03:59,560 without giving us their password? 48 00:03:59,560 --> 00:04:04,560 First, rather than give us their password, instead give us a credential that's unique to us, 49 00:04:04,560 --> 00:04:08,560 and tell the third-party application, which is the accounts package in this example, 50 00:04:08,560 --> 00:04:12,560 to only let us have access to the particular information we need access to. 51 00:04:12,560 --> 00:04:15,560 So, our application will have scoped permission. 52 00:04:15,560 --> 00:04:19,560 This means, for example, our application has access to invoices and customers, 53 00:04:19,560 --> 00:04:22,560 but not payroll or other ledger accounts. 54 00:04:22,560 --> 00:04:25,560 If they're no longer one, our status is their data, 55 00:04:25,560 --> 00:04:28,560 then they do not have to change the password, 56 00:04:28,560 --> 00:04:32,560 and said they simply tell the accounts package to provide our access. 57 00:04:32,560 --> 00:04:38,560 So, we've asked the user to get something that stands in place for password from their accounts package. 58 00:04:38,560 --> 00:04:40,560 This is commonly known as an API key. 59 00:04:40,560 --> 00:04:43,560 I've talked more about the meaning of API later. 60 00:04:43,560 --> 00:04:48,560 This situation is better, because now we don't have the user's actual password, 61 00:04:48,560 --> 00:04:50,560 and we have more limited access. 62 00:04:50,560 --> 00:04:56,560 But still, there is a high risk that over a long period of time, this secret key could get leaked. 63 00:04:56,560 --> 00:04:59,560 It would be nice if the secret changed often, 64 00:04:59,560 --> 00:05:05,560 and ideally changing the secret should be an automated process to avoid hassle to our users. 65 00:05:05,560 --> 00:05:10,560 So, to reiterate, we want our application to get to temporary token, 66 00:05:10,560 --> 00:05:14,560 which is like a password, except that it's randomly generated, 67 00:05:14,560 --> 00:05:16,560 and only valid for limited amount of time. 68 00:05:16,560 --> 00:05:20,560 When it expires, we should have the means to get a new token. 69 00:05:20,560 --> 00:05:23,560 The process I've described has been standardised, 70 00:05:23,560 --> 00:05:26,560 and is known as OAuth2, 71 00:05:26,560 --> 00:05:30,560 which is OAuth8, and the number 2. 72 00:05:30,560 --> 00:05:34,560 What does this look like in practice for the end user? 73 00:05:34,560 --> 00:05:36,560 Well, the user starts off in our application, 74 00:05:36,560 --> 00:05:41,560 and sees a button that says authorised connection to QuickBooks or Zero or whatever. 75 00:05:41,560 --> 00:05:45,560 They click it, and then they take into their accounts package website. 76 00:05:45,560 --> 00:05:48,560 They love it into their accounts package. 77 00:05:48,560 --> 00:05:52,560 It asks them, do you want to allow this company to have access to a data? 78 00:05:52,560 --> 00:05:56,560 And it will list the scope of the access, for example, reading right, 79 00:05:56,560 --> 00:05:59,560 and which data in particular can be accessed. 80 00:05:59,560 --> 00:06:03,560 Assuming they click OK, they'll be redirected back to our application. 81 00:06:03,560 --> 00:06:05,560 And how does this actually work? 82 00:06:05,560 --> 00:06:09,560 Well, we first register our app with the accounts package. 83 00:06:09,560 --> 00:06:12,560 We turn on which URL we want them to redirect back to, 84 00:06:12,560 --> 00:06:15,560 when one of their users has given us access to their data. 85 00:06:15,560 --> 00:06:21,560 The accounts package allocates us some credentials to use when we communicate with them. 86 00:06:21,560 --> 00:06:27,560 We construct URL that will take the user from our application to the accounts package for authorization. 87 00:06:27,560 --> 00:06:32,560 When user has clicked on the URL, and authenticated with the accounts package, 88 00:06:32,560 --> 00:06:38,560 an authorized application to access their data, then users browse a redirects back to our website, 89 00:06:38,560 --> 00:06:42,560 passing a code in the query string of the URL. 90 00:06:42,560 --> 00:06:47,560 Our application must immediately use this code to request a token from the accounts package, 91 00:06:47,560 --> 00:06:51,560 which will then store associated with that user. 92 00:06:51,560 --> 00:06:53,560 In fact, we get two tokens. 93 00:06:53,560 --> 00:06:59,560 One of them pulled the access token is what will be using to authenticate every request we make to the accounts package. 94 00:06:59,560 --> 00:07:03,560 The other token is what's called a refresh token. 95 00:07:03,560 --> 00:07:07,560 This one is only used to get a fresh access token when it expires. 96 00:07:07,560 --> 00:07:11,560 Since typically an access token is only valid for about 24 hours say. 97 00:07:11,560 --> 00:07:16,560 These tokens are usually very long and apparently random strings of characters. 98 00:07:16,560 --> 00:07:20,560 They may in fact encode some readable information, but for our purposes, 99 00:07:20,560 --> 00:07:27,560 we should treat them as opaque random blocks of data, which are needed to authenticate. 100 00:07:27,560 --> 00:07:33,560 The crests I've mentioned are all standard HTTPS requests, which is the same protocol 101 00:07:33,560 --> 00:07:38,560 which web grass is used to talk to a web server when it downloads web pages. 102 00:07:38,560 --> 00:07:42,560 HTTPS stands for Hypertext Transfer Protocol. 103 00:07:42,560 --> 00:07:49,560 The S stands for Secure, which is important as these messages contain secret tokens, so I should be encrypted. 104 00:07:49,560 --> 00:07:53,560 The format of the data we exchange is typically JSON. 105 00:07:53,560 --> 00:08:00,560 That is JSON, standing for JavaScript object notation, which is a simple human readable format 106 00:08:00,560 --> 00:08:05,560 for encoding data consisting of a few primitive data types like strings and numbers, 107 00:08:05,560 --> 00:08:09,560 as well as arrays and objects that can be nested inside each other. 108 00:08:10,560 --> 00:08:16,560 Once we have our access token, we can begin making requests to the third party application. 109 00:08:16,560 --> 00:08:21,560 Again, these are HTTPS requests, and the data format is usually JSON, 110 00:08:21,560 --> 00:08:25,560 or sometimes another human readable format called XML. 111 00:08:25,560 --> 00:08:30,560 To authenticate these requests, we would include a header with our request, 112 00:08:30,560 --> 00:08:36,560 that says authorization, colon, space, bearer, then space followed by our token. 113 00:08:36,560 --> 00:08:42,560 So now we are able to talk to our users account package, but how do we know what to say? 114 00:08:42,560 --> 00:08:45,560 This is where APIs come in. 115 00:08:45,560 --> 00:08:50,560 API stands for application programming interface, and it's simply a standard set of commands, 116 00:08:50,560 --> 00:08:56,560 and their parameters along with definitions for data structures used to communicate information. 117 00:08:56,560 --> 00:09:00,560 A common pattern for APIs on the internet is rest. 118 00:09:00,560 --> 00:09:04,560 The acronym rest stands for representational state transfer. 119 00:09:04,560 --> 00:09:09,560 Basically, all it means is we should use standard HTTP requests for sending, 120 00:09:09,560 --> 00:09:13,560 receiving data using same verbs already defined for HTTP, 121 00:09:13,560 --> 00:09:18,560 which are get, post, put, patch, and delete. 122 00:09:18,560 --> 00:09:22,560 By having a meaningful path in the URL, we can send requests such as, 123 00:09:22,560 --> 00:09:28,560 get, forward slash invoices, or post, forward slash invoices, 124 00:09:28,560 --> 00:09:32,560 or delete, forward slash invoices, forward slash 1234. 125 00:09:33,560 --> 00:09:40,560 These requests will get all invoices, add a new invoice, or delete a specific invoice respectively. 126 00:09:40,560 --> 00:09:45,560 Now I've described the what and why of connecting services, authenticating with our 127 00:09:45,560 --> 00:09:48,560 author and getting two different applications to talk to each other. 128 00:09:48,560 --> 00:09:52,560 What about the how? What tools are available to help with this? 129 00:09:52,560 --> 00:09:57,560 When you're playing around with rest APIs, a good tool to use is postman. 130 00:09:57,560 --> 00:10:00,560 This has a nice user interface but putting together requests. 131 00:10:00,560 --> 00:10:06,560 You can also go old school and use a command line tool like curl to send HTTP requests. 132 00:10:06,560 --> 00:10:11,560 And there are also web-based applications that simplify the process of using APIs, 133 00:10:11,560 --> 00:10:16,560 which are SAP-A. Most popular applications will have their own plugins, 134 00:10:16,560 --> 00:10:22,560 but SAP-A, so integrating two services can be as simple as authenticating with them 135 00:10:22,560 --> 00:10:27,560 by then defining what particular information you want to be synchronized. 136 00:10:27,560 --> 00:10:33,560 Also, lots of vendors that provide an API will also provide code libraries for popular languages 137 00:10:33,560 --> 00:10:39,560 that can be used to create an integration without writing the whole thing from scratch. 138 00:10:39,560 --> 00:10:46,560 Every vendor who has an API and actually wants people to use it will make sure it's documented well on the website. 139 00:10:46,560 --> 00:10:54,560 There are standard ways of documenting APIs such as if an API and tools that can interface with a specification format. 140 00:10:55,560 --> 00:11:02,560 So I've talked about a few approaches for connecting between two web applications with an example of a familiar way for which it's accounting packages. 141 00:11:02,560 --> 00:11:08,560 There are some variations in what I've described. For example, if the app is a mobile app with no backend, 142 00:11:08,560 --> 00:11:17,560 then while still using OAuth2, the flow will be slightly different. You'll probably be working with a signed block of data known as a JSON web token. 143 00:11:17,560 --> 00:11:21,560 Practically, this means rather than having what's known as offline access, 144 00:11:22,560 --> 00:11:27,560 the app professional have to ask the user to reauthenticate whenever communicating with the third party. 145 00:11:27,560 --> 00:11:32,560 For more about OAuth2, you can look at OAuth.net or many other online resources. 146 00:11:32,560 --> 00:11:39,560 So many popular web applications do provide an API and it really is useful being able to work with these. 147 00:11:39,560 --> 00:11:49,560 If you have some coding skills and want to get started accessing APIs, then services that use an API key would probably be the ones to start with since OAuth2 148 00:11:49,560 --> 00:11:54,560 is somewhat more complicated than not terrible once you get your head around it. 149 00:11:54,560 --> 00:12:07,560 If you just want to play around and want to see what public APIs are out there, a good place to start is github.com for slash public hyphen APIs for slash public hyphen APIs. 150 00:12:07,560 --> 00:12:16,560 Even if you're not writing your own integration, you're likely to already be granting different third party apps access to the web based services you use, 151 00:12:16,560 --> 00:12:23,560 which are using off to behind the scenes whenever you do this. Either way, I hope this has been of some interest. Thanks for listening. 152 00:12:46,560 --> 00:12:51,560 An onsthost.com, the internet archive and our synced.net. 153 00:12:51,560 --> 00:12:56,560 On the satellite stages, today's show is released on our creative comments. 154 00:12:56,560 --> 00:13:00,560 Attribution for.0 international license.